N95- 23745 


Automated Synthesis of Image Processing 
Procedures Using AI Planning Techniques 

Steve Chien and Helen Mortensen 

Jet Propulsion Laboratory, 

California Institute of Technology 
4800 Oak Grove Drive 
Pasadena, CA 91 109-8099 

Tel: (818)306-6144 Fax: (818)306-6912 
E-Mail: chien@aig.jpl.nasa.gov 


KEY WORDS AND PHRASES 

Automation, planning, science data analysis, large 
databases, science operations. 

Introduction 

This paper describes the Multimission VICAR 
(Video Image Communication and Retrieval 1 ) 
Planner (MVP) (Chien 1994) system, which uses 
artificial intelligence planning techniques (Iwasaki 
& Friedland, 1985, Pemberthy & Weld, 1992, 
Stefik, 1981) to automatically construct executable 
complex image processing procedures (using 
models of the smaller constituent image processing 
subprograms) in response to image processing 
requests made to the JPL Multimission Image 
Processing Laboratory (MIPL). The MVP system 
allows the user to specify the image processing 
requirements in terms of the various types of 
correction required. Given this information, MVP 
derives unspecified required processing steps and 
determines appropriate image processing programs 
and parameters to achieve the specified image 
processing goals. This information is output as an 
executable image processing program which can 
then be executed to fill the processing request. 

Currently, a group of human experts, called 
analysts, receive written requests from scientists for 
image data processed and formatted in a certain 


* This work was performed by the Jet Propulsion 
Laboratory, California Institute of Technology, under 
contract with the National Aeronautics and Space 
Administration. Other past and present members of the 
MVP team are Christine Ying, Shouyi Hsiao, Alex Gray, 
Joe Nieten, and Jean Lorre. 

1 This name is somewhat misleading as VICAR is used 
to process considerable non-video image data such as 
MAGELLAN synthetic aperture radar data. 


manner. These analysts then determine the relevant 
data and appropriate image processing steps 
required to produce the requested data and write an 
image processing program in a programming 
language called VICAR (LaVoie et al.1989). 

Unfortunately, this current mode of operations 
is extremely labor- and knowledge-intensive. This 
task is labor intensive in that constructing the image 
processing procedures is a complex, tedious process 
which can take up to several months of effort. 
There are currently tens of analysts at MIPL alone 
whose primary task is to construct these VICAR 
programs. Many other users at JPL and other sites 
also write VICAR scripts, with the total user group 
numbering in the hundreds. 

The VICAR procedure generation problem is 
also a knowledge-intensive task. In order to 
construct VICAR procedures, an analyst must 
possess knowledge of: 

1. image processing and image processing 
programs (as of 1/93 there were 
approximately 50 frequently used 
programs, some having as many as 100 
options) 

2. database organization and database label 
information to understand the state of 
relevant data 

3. the VICAR programming language to 
produce and store relevant information. 

Because of the significant amount of 
knowledge required to perform this task, it takes 
several years for an analyst to become expert in a 
VICAR image processing area. 

The MVP task targets automated generation of 
image processing procedures from user requests and 
a knowledge-based model of an image processing 
area using artificial intelligence (AI) automated 
planning techniques. In AI planning, a system uses: 
1 ) a model of actions in a domain; and 2) a model 
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of the current state to reason about what actions to 
take to achieve some specified goals. By partially 
automating the filling of basic science requests, 
request turnaround time will be reduced, analysts 1 
time will be freed for more complex and 
challenging science requests, and analysts’ 
workload will be reduced. 

VICAR is a general-purpose image processing 
programming language designed to promote the 
development and re-use of general-purpose image 
processing algorithms for MIPL needs. The 
primary function of VICAR is to allow individual 
image processing steps (called VICAR programs) to 
be combined into more complex image processing 
scripts called procedure definition files (PDFs). As 
one of their primary duties, MIPL analysts construct 
PDFs to perform image correction, image 
enhancement, construct mosaics, and to create 
movies and render objects. Individual processing 
programs perform functions such as: 

photometric correction - correcting the image 
for lighting conditions due to the position of the sun 
relative to the imaging device and target, 

radiometric correction - correcting for varying 
camera response depending on where in the field of 
view the image is read, 

line fill-in - interpolating missing lines caused 
by data transmission errors. 

By composing individual programs which 
perform these specialized functions, analysts can 
create complex image processing procedures 
(PDFs) to perform multiple types of correction and 
register the images to allow combination of 
multiple images into larger images. 

The MVP Architecture 

The overall architecture for the MVP system is 
shown in Figure 1. The user inputs a problem 
specification consisting of processing goals and 
certain image information using a menu-based 
graphical user interface. These goals and problem 
contexts are then passed to the decomposition-based 
planner which uses skeletal and hierarchical 
planning methods to classify the problem type and 
then uses this classification to decompose the 
problem into smaller subproblems. During this 
decomposition process, MVP determines which 
information on the database state is needed by the 
planner to solve the subproblems. 

These subproblems are then solved by a 
conventional operator-based planner that uses the 
subproblem goals and initial states as indicated by 
the problem decomposition. The resulting plan 
segments are then assembled using constraints 
derived in the decomposition process. The resulting 
plan is then used to generate an actual executable 


VICAR PDF using conventional macro-expansion 
techniques. 



Figure 1: MVP Architecture 

Plans in the MVP domain can be of 
considerable length (up to 100 steps) and each step 
(or VICAR program) can involve reasoning about 
numerous complex effects (many operators have 
tens of effects). Due to the large search space 
caused by this complexity, conventional operator- 
based planning approaches are not able to tractably 
construct plans in the VICAR domain without 
significant control knowledge. 

Additionally, even if a purely operator-based 
planning approach were able to generate plans to 
solve the VICAR problems, these plans would be 
difficult for MIPL analysts to understand. 
Typically, analysts begin by classifying the general 
problem being addressed into one of a general class 
of problems, such as mosaicking, color triple 
processing, etc. They then use this classification 
and the problem context to decompose the plan into 
several abstract steps, such as local correction, 
navigation, registration, touch-ups, etc. A planning 
system which mimicked this approach to producing 
VICAR PDFs would be desirable. 

Skeletal and Hierarchical Planning Using 
Decompositions in MVP 

Skeletal planning (Iwasaki & Friedland 1985) is 
an approach to planning which casts planning as a 
structured classification problem. In skeletal 
planning, a planner identifies a new problem as one 
of a general class of problems, based upon the goals 
and initial state. This technique was originally 
developed as a model of experiment design in 
molecular biology; however, skeletal planning is 
also an accurate model of how expert analysts 
attack VICAR procedure generation problems. 
Typically, in a VICAR problem, there is a central 
goal for processing, such as mosaicking, which then 
dictates a decomposition of the overall problem into 
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subproblems such as local correction, navigation, 
and registration. MVP attacks a VICAR problem 
by first determining the general problem class, and 
then using this problem class to perform an initial 
decomposition of the top-level image processing 
goals. 

Hierarchical planning (Stefik 1981) is an 
approach to planning where abstract goals or 
procedures are incrementally refined into more and 
more specific goals or procedures as dictated by 
goal or procedure decompositions. MVP uses this 
approach of hierarchical decomposition to refine the 
initial skeletal plan into a more specific plan which 
has been specialized, based on the specific current 
goals and situation. This allows the overall 
problem decomposition to be influenced by factors 
such as the presence or absence of certain image 
calibration files or the type of instrument and 
spacecraft used to record the image. For example, 
geometric correction uses a model of the target 
object to correct for variable distance from the 
instrument to the target. For Voyager (VGR) 
images, geometric correction is performed as part of 
the local correction process, as geometric distortion 
is significant enough to require immediate 
correction before other image processing steps can 
be performed. However, for Galileo (GLL) images, 
geometric correction is postponed until the 
registration step, where it can be performed more 
efficiently. 

MVP uses a decomposition-based approach 
(Lansky 1993) to perform Skeletal and Hierarchical 
planning. In a decomposition-based approach, 
decomposition rules dictate how in plan-space 
planning, one plan can be legally transformed into 
another plan. The planner then searches the space 
plans defined by these decompositions. 
Decomposition-based approaches are extremely 
powerful in that many other paradigms, such as 
modal truth criterion planning (Lansky 1993), can 
be implemented in a decomposition-based 
approach. 

This decomposition-based approach to skeletal 
and hierarchical planning in MVP has several 


strengths. First, the decomposition rules very 
naturally represent the manner in which the analysts 
attack the procedure generation problem. Thus, it 
was a relatively straightforward process to get the 
analysts to articulate and accept classification and 
decomposition rules for the subareas which we have 
implemented thus far. Second, the notes from the 
decomposition rules used to decompose the 
problem can be used to annotate the resulting PDF 
to make the VICAR programs more understandable 
to the analysts. Third, relatively few problem 
decomposition rules are easily able to cover a wide 
range of problems and decompose them into much 
smaller subproblems. 

Operator-based Planning in MVP 

MVP uses classical operator-based planning 
techniques to solve subproblems produced by the 
decomposition-based planner. An operator-based 
planner uses: 1 . a model of actions, A (in this case 
the model represents the requirements and effects of 
individual VICAR steps); 2. a specification of a 
current state, C (this corresponds to the current 
database state); and 3. a specification of a goal 
criterion, G (this corresponds to user request specifi- 
cation), to derive a sequence of actions, A', that when 
executed in the current state C, results in a state which 
satisfies the goal criterion G. 

To illustrate this process, consider the following 
5 simplified image processing operators shown in 
Figure 2. Preconditions are attributes which must be 
true of the image file before the step can be run, and 
effects are attributes which are made true by 
executing the step. This information can be 
summarized by the information shown below 
indicating the relevant programs for achieving the 
goals of missing line fill-in, spike removal, and 
radiometric correction for Voyager and Galileo 
images. When constructing a plan to achieve these 
goals, depending on the project of the image file 
(e.g., either Voyager or Galileo), MVP will know 
the correct program to use because the 
preconditions enforce the correct program selection. 


Operator VGRFILLIN GLLFILLIN ADESPIKE 


FICOR77 GALSOS 


Preconditions VGR image GLL image 
EDR (binary 
header) present 

Effects missing lines filled in 


(GLL image) 
or ((VGR image) 
and (raw values)) 
spike removal 


VGR image GLL image 

raw pixel values 


radiometric corr. radiometric corr. 

blemish removal Reed-Solomon 

overflow corr. 

not raw values saturated pixel corr. 

not missing line fill-in 


Figure 2: Simplified Planning Operators 
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However, determining the correct ordering of 
actions can sometimes be complex. In this case, the 
correct order to achieve the goals of line fill-in, 
spike removal, and radiometric correction is 
dependent upon the project of the file. In the case 
of Voyager files, ADESPIKE (spike removal) 
requires raw pixel values, and FICOR77 
(radiometric) changes pixel values to correct for 
camera response function, so FICOR77 removes a 
necessary condition for ADESPIKE. This 
interaction can be avoided by requiring that 
ADESPIKE occur before FICOR77. VGRFILLIN 
requires a binary EDR header on the image file which 
is not maintained by ADESPIKE, this interaction 
can be avoided by requiring VGRFILLIN to be 
executed before ADESPIKE. 


The Galileo case is slightly different. GALSOS 
undoes missing line fill-in so that it interferes with 
GLLFILLIN. This interaction can be avoided by 
enforcing GLLFILLIN after GALSOS. 
Additionally, GALSOS requires raw pixel values, 
and ADESPIKE alters the pixel values, so 
ADESPIKE interferes with this condition. This 
interaction can be avoided by requiring that 
GALSOS occur before ADESPIKE. 

Voyager Galileo 

fill-in missing lines VGRFILLIN GLLFILLIN 

remove spikes ADESPIKE ADESPIKE 

radiometric corr. FICOR77 GALSOS 

Execution Order: VGRFILLIN GALSOS 

ADESPIKE GLLFILLIN 

FICOR77 ADESPIKE 


This simple example illustrates the types of 
interactions and context-sensitivity that the VICAR 
image processing application entails. All of these 
interactions and context sensitive requirements are 
derived and accounted for automatically by MVP 
using the operator specification, thus allowing 
construction of plans despite complex interactions 
and conditions. 


Current Status and Conclusions 

MVP is currently operational and in use by 
analysts at JPL's Multimission Image Processing 
Laboratory (MIPL). Over a test suite of 5 typical 
mosaicking and color reconstruction tasks, an 
expert analyst estimated that MVP reduces effort to 


generate an initial PDF for an expert analyst from 
1/2 a day to 15 minutes, and that it would reduce 
the effort for a novice analyst from several days to 1 
hour. 

MVP uses a combination of decomposition- 
based and operator-based planning paradigms to 
substantially automate the process of generating 
image processing procedures for radiometric 
correction and color triplet reconstruction. Current 
efforts involve expanding MVP to cover areas in 
filtering, stretching, and more complex relative 
navigation tasks. 
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